Bootsrapping Hypothesis Testings in Credit Risk

Andrija Djurovic

Bootstrapping in Credit Risk

- Generally underutilized in the credit risk area. Particularly useful for testing metrics where there is little consensus on standard error or when statistical testing procedures are absent.

Advantages:
- does not rely on any distributional assumption
- flexible application across a wide range of metrics
- useful for identifying bias
- easy to implement.
Disadvantages:
- computational intensity
- assumption of independence (although this can be addressed to a certain extent)
- accuracy concerns for smaller sample sizes.

Example 1: Population Stability Index (1-sided test)

Dataset:

##   Bin Base Target  PSI
## 1   1 0.22   0.35 0.18
## 2   2 0.24   0.31 0.18
## 3   3 0.07   0.05 0.18
## 4   4 0.48   0.29 0.18

Testing Hypothesis:
What is the probability that PSI value is less or equal to 0.15?

Visualization:

## p-value = 21.26%

Example 2: Herfindahl-Hirschman Index (1-sided test)

Dataset:

##         Rating Grade # obs.   DR
## 1   01 (-Inf,0.0199)    202 0.01
## 2 02 [0.0199,0.0263)     54 0.02
## 3 03 [0.0263,0.0369)     96 0.03
## 4 04 [0.0369,0.0903)    204 0.06
## 5   05 [0.0903,0.15)    103 0.11
## 6    06 [0.15,0.197)     41 0.12
## 7     07 [0.197,Inf)     50 0.32

Testing Hypothesis:
What is the probability that HHI value is greater or equal to 0.20?

Visualization:

## p-value = 23.72%

Example 3: Area Under Curve (2-sided test)

Dataset:

## Bootstrapped AUC summary:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.6769  0.7414  0.7526  0.7525  0.7638  0.8067

## Development sample AUC 79%.

## Application portfolio AUC 75.2%.

Testing Hypothesis:
What is the probability that the application portfolio AUC is equal to 79%?

Figure 1.

Figure 2.

## p-value = 2.26%